A comparison and integration of capture-recapture models and the detection profile method

In order to control inspections, the number of remaining defects in software artifacts after their inspection should be estimated. This would allow, for example, deciding whether a reinspection of supposedly faulty artifacts is necessary. Several studies in software engineering have considered capture-recapture models for performing such estimations. These models were initially developed for estimating animal abundance in wildlife research. In addition to these models, researchers in software engineering have recently proposed an alternative approach, namely the Detection Profile Method (DPM), that makes less restrictive assumptions than some capture-recapture models and that show promise in terms of estimation accuracy. In this study, we investigate how to select between these two approaches for defect content estimation. As a result of this investigation we present a selection procedure taking into account the strength and weaknesses of the two methods. A weakness known for capture-recapture models is that they tend to provide extreme under/over estimation. The existence of such extreme outliers can discourage their use because their consequences in terms of wasted effort or defect slippage can be substantial, and therefore it is not clear whether a particular estimate can be trusted. The evaluation of our selection procedure with actual inspection data indicates that this selection procedure provides the same accuracy as capture-recapture models alone and DPM alone, and most importantly does not exhibit extreme over/under estimation. Thus, this selection procedure can be used in practice with a high degree of confidence since its estimates are not likely to exhibit extreme estimation error.